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Abstract. A nonnegative tensor has nonnegative rank at most 2 
if and only if it is supermodular and has flattening rank at most 2. 
We prove this result, then explore the semialgebraic geometry of 
the general Markov model on phylogenetic trees with binary states, 
and comment on possible extensions to tensors of higher rank. 



1. Introduction 

This article offers a journey into semialgebraic statistics. By this we 
mean the systematic study of statistical models as semialgebraic sets. 
We shall give a semialgebraic description of binary latent class models 
in terms of binomials expressing supermodularity, and we determine 
the algebraic boundary of this and related models. Our discussion is 
phrased in the language of nonnegative tensor factorization [5, 9]. 

We consider real tensors P = [pi 1 i 2 ---i n ] of format d\ x d 2 x ■ ■ ■ x d n . 
Throughout this paper we shall assume that n > 3 and d\, g?2> • • • , d n > 
2. Such a tensor has nonnegative rank at most 2 if it can be written as 

(1) P = a\ <g> a 2 <E> • • • g) a n + hi <g> b 2 <8> • • • <8> b n , 

where the vectors a,, hi G Mr* are nonnegative for % = 1,2, ... ,n. The 
set of such tensors is a closed semialgebraic subset of dimension 2(di + 

rf 2 H h4)-2(n-l) in the tensor space Rd 1 xd 2 x-xd„. gee [ 13; ^5 5] 

We present the following characterization of this semialgebraic set. 

Theorem 1.1. A nonnegative tensor P has nonnegative rank at most 
2 if and only if P is supermodular and has flattening rank at most 2. 

Here, flattening means picking any subset A of [n] = {1,2, ... ,n} 
with 1 < \A\ < n — 1 and writing the tensor P as an ordinary matrix 
with riieA rows an d TljtgA dj columns. The flattening rank of P is 
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the maximal rank of any of these matrices. Landsberg and Manivel 
[14] proved that flattening rank < 2 is equivalent to border rank < 2. 

To define supermodularity, we first fix a tuple n = (tti, 7r 2 , . . . , n n ) 
where 7Tj is a permutation of {1, 2, . . . , di}. Then P is it -supermodular if 

(2) Vi\ii---in ' Pjij2---jn — Pkik2---k n ■ Phh — ln 

whenever {i r ,j r } = {K,l r } and7r r (fc r ) < ir r (l r ) holds for r = 1, 2, . . . , n. 
We call a tensor P supermodular if it is 7r-supermodular for some n. 

Note that we are using multiplicative notation instead of the additive 
notation more commonly used for supermodularity. To be specific, if 
di = d 2 = ■ ■ ■ = d n = 2, 7r = (id, id, . . . , id), and P is strictly positive, 
then P being 7r-supermodular means that log(P) lies in the convex 
polyhedral cone [16, §4] of supermodular functions 2"^ 1,2, "•'"^ — > IR. 

The set of 7r-supermodular nonnegative tensors P of flattening rank 
< 2 is denoted M.^ and called a toric cell. The number of toric cells is 
d\\d 2 \ ■ ■ ■ d n \/2. Theorem 1.1 states that these cells stratify our model: 

(3) M = U^Mk- 

The term model refers to the fact that intersection of (3) with the 
probability simplex, where all coordinates of P sum to one, is a widely 
used statistical model. It is the mixture model for pairs of independent 
distributions on n discrete random variables. This justifies our earlier 
claims about the dimension of M. and that it is topologically closed. 

Recall that the Zariski closure S of a semialgebraic subset S of IR^ 
is the zero set of all polynomials that vanish on S. The boundary OS 
is the topological boundary of S inside S. We define the algebraic 
boundary of S to be the Zariski closure dS of its topological boundary. 

Our second theorem concerns the algebraic boundaries of the model 
M. and of toric cells M.^. We regard these boundaries as hypersurfaces 
inside the complex variety of tensors of border rank < 2. A slice of our 
tensor P is a subtensor of some format d\X- ■ -xd s ^ixlxd s+ ix- ■ -xd n . 
Subtensors of format d\X - ■ ■x<i 5 _ 1 x2x<i s+1 x- ■ -xd n are double slices. 

Theorem 1.2. The algebraic boundary of Ai has Y^7=i^i irreducible 
components, given by slices having rank < 1. The algebraic boundary 
of any toric cell A4 V has the same irreducible components, but it has 
J^r=i O2 ) f ur "^her components, given by linearly dependent double slices. 

A double slice is linearly dependent if its two slices are identical up to 
a multiplicative scalar. In the second component count of Theorem 1.2 
we exclude the special case 2x2x2 because the "further components" 
fail to be hypersurfaces. If n = 2 then the rank 1 constraint on slices 
is void, and the algebraic boundary consists of the d\d 2 coordinate 
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hyperplanes in IR dlXd2 . This is consistent with the fact [7, Example 
4.1.2] that all nonnegative matrices of rank 2 have nonnegative rank 2. 

This paper is organized as follows. In Section 2 we derive our two 
theorems for tensors of format 2x2x2. This extends results in [3, 
4, 12, 17, 21] on this widely studied latent class model. Here, our 
semialgebraic set Ai is full-dimensional in M 2x2x2 , and it consists of 
four toric cells that are glued together. Any two cells intersect along 
the locus where one of the flattenings has rank one. The common 
intersection of all cells is the independence model (tensors of rank 1). 
In Section 3 we prove Theorems 1.1 and 1.2 for arbitrary di, d 2 , . . . , d n . 

The set Ai above appears in phylogenetics as the general Markov 
model on a star tree with binary states. Section 4 develops the exten- 
sion of our results to phylogenetic trees other than star trees. These 
models have another type of component in their algebraic boundaries, 
characterized by the constraint that the ranks of certain matrix flatten- 
ings of P inconsistent with the tree topology drop from 4 to 3. Super- 
modularity in this context was pioneered by Steel and Faller [20] . Our 
results refine earlier work on the general Markov model in [2, 3, 12, 21]. 

Section 5 concerns the challenges to be encountered when trying to 
extend our results to tensors of higher rank. We present case studies 
of algebraic boundaries for one identifiable model (3 x 3 x 2-tensors of 
rank 3) and one non-identifiable model (2 x 2 x 2 x 2-tensors of rank 3). 



2. The Base Case 

Let P = [pijk] be a real 2x2x2 tensor. Then P has nonnegative 
rank at most 2 if there exist three nonnegative 2 x 2-matrices 



hi 



Ol2 



A, 



0-21 &22 
021 022 



and A-i 



«31 O32 
^31 O32 



such that 

(4) p^k = aiia 2 ja 3k + biib 2 jh k for k 6 {1, 2}. 

For it = (id, id, id), the binomial inequalities for supermodularity are 



P111P222 > P112P221 
(5) P112P222 > P122P212 
P111P122 > P112P121 



P111P222 > P121P212 
P121P222 > P122P221 
P111P212 > P112P211 



P111P222 > P211P122 
P211P222 > P212P221 
P111P221 > P121P211 



Nonnegative 2x2x2 tensors P that satisfy these nine inequalities lie 
in the toric cell A^id,id,id = M- (12), (12), (12)- By label swapping 1 -h- 2, 
we obtain three other toric cells A4 id ,id,(i2) = A4(i 2 ),(i2),id, -M id ,(i2),id = 
A4(i2),id,(i2), and «M(i2),id,id = Md,(i2),(i2)- Thus, by definition, the 
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semialgebraic set of all supermodular 2 x 2 x 2-tensors is the union 

(6) M = McMd.id U M d) id,(i2) u - A ^id,(i2),id U -M ( i2),id,id- 

Theorem 1.1 states that P G j^ 2 * 2 * 2 hag nonnegative rank at most 2 if 
and only if P lies in A4. We begin by proving the only- if direction. 

Lemma 2.1. If P G ]R 2x2x2 has nonnegative rank at most 2, then P 
is supermodular. More precisely, define it = (7Ti, 7T2, 7r 3 ) by Hi = id if 
det(A) > and vr; = (12) if det(Ai) < 0. Then P G M v . 

Proof. Let P be as in (4). The last six constraints in (5) specialize to 

P112P222 - P122P212 = a32&32det(v4i)det(v4 2 ), 

P121P222 - P122P221 = a 22 & 22 det(v4i)det(v4 3 ), 

P211P222 - P212P221 = ai2&i2det(y4 2 )det(^ 3 ), 

' P111P122 - P112P121 = aii6iidet(A 2 )det(i4 3 ), 

P111P212 - P112P211 = a 2 i&2idet(v4 1 )det(v4 3 ), 

P111P221 - P121P211 = a 3 i&3idet(v4 1 )det(v4 2 ). 

First suppose that all 12 parameters and 6^ and the three determi- 
nants det(-Afc) are positive. Then the six expressions in (7) are positive. 
The first three constraints in (5) are also satisfied, as seen from 

/ g \ P111P222 - P121P212 = 

(Plll(Pll2P222 - P122P212) + P212(PlllPl22 ~ Pll2Pl2l))/pil2- 

Second, consider all tensors P where the parameters Oy , 6^ and de- 
terminants det(Afc) are nonnegative. These lie in the closure of the 
previous case, so the nine binomials will be nonnegative. 

Next observe that nP = (p^- 1 ^- 1 ^-)^ 1 ^)) a ^ so nas nonnegative 
rank < 2, with parameterization given by swapping the columns of A, 
whenever 7Tj = (12). This changes the sign of detA{. Hence, P is in 
M. n if and only if 7r _1 P is in A^id,id,id- D 

We now prove Theorem 1.1 for 2 x 2 x 2 tensors. In this special case, 
the flattening rank is automatically < 2, so there are no equational 
constraints, and our model Ai is a full-dimensional subset of M 2x2x2 . 

Proposition 2.2. Let P be a nonnegative 2 x 2 x 2-tensor. Then P 
has nonnegative rank < 2 if and only if P is supermodular. 

Proof. If P has nonnegative rank < 2, then P is supermodular by 
Lemma 2.1. For the converse, suppose that P is supermodular. Define 

U12 = P11+P22+ ~ P12+P21+, 
(9) U 13 = P1+1P2+2 ~ P1+2P2+1, 

U23 = P+11P+22 -P+12P+21, 
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where a subscript + refers to summing over all values of the given 
index. For example, P22+ = P221 + ^222- Similarly, for i — 1,2, define 



Our strategy is to first show that the following hold for P: 

(i) u 12 u l3 u 23 > 0, 

(ii) Uh and [/?■ have the same sign as Uj for every i < j, and 

(iii) if = 0, then Ulj = U* = 0. 

Subsequently, in the second step, we will show that (i), (ii), (iii) imply 
that P has the form (4) with Ai,A 2 , A3 nonnegative. That second step 
will follow proofs of closely related results in [3, 4, 12, 17, 21]. 

Let e = (id, id, id). Since nM. e = Ai n , and the conditions (i),(ii),(iii) 
are invariant under label swapping, it suffices to consider P e M e . By 
definition of e-supermodularity, U^, Z7?- > 0. We need to show that 
U^ > also. By symmetry it suffices to show that U\ 2 > 0. We have 

(11) U 12 = U\ 2 + Ul 2 + (pmP222 + P221P112 - P211P122 - P121P212)- 

We show that the expression in parentheses is nonnegative for P £ Ai e . 
We write this expression as R = fn + f 22 — /12 — /21, where 

fll = P111P222, /l2 = P121P212, /21 = P211P122, /22 = P221P112 

Note that, by (5), we have fn > max{/ 12 , /2i}- This implies R > 
if either p^f. = for some i,j,k, or if f 22 > min{/i 2 , /ai}- Thus, 
we assume that > and f 22 < min{/ 12 , f 2 \}. The supermodular 
inequalities P121P211 < P111P221 and P212P122 < P222P112 imply 

/12/2I = Pl2lP212P21lPl22 < PIHP222P221PH2 = /ll/22- 

Hence [/y] is supermodular itself. As a consequence, we have 



or equivalently R > 0. It follows that C/12 > and, by symmetry, that 
U^ > for all z < j; therefore (i) and (ii) hold. The identity (11) and 
the inequality R > together imply that (iii) holds as well. 



(10) 



U$ 2 = PlliP22i - Pl2iP21i, 
U{ 3 = P\i\P2i2 -PU2P2H, 

U 23 = PiixPm - ViviPnx- 




where the second inequality holds since f 22 < f\ 2 < fn. 
After multiplying both sides by fn we obtain 



/21 — fll < /22 — fl2 
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We take up separately the cases where the product U12U13U23 is 
positive or is zero. Suppose first that U12U13U23 > 0. A special role in 
our argument will be played by the hyperdeterminant, 

Det(P) = 4p ni p 12 2P212P221 + 4pn2Pl2lP21lP222 + P?nP222 + ^122^211 
+P112P221 _ 2PU1PH2P221P222 - 2j9mPi2lP212P222 - 2pmPi22£>21lP222 
+Pl2lP212 _ 2pu 2 p 12 lP212P221 - 2pn2Pl22P21lP221 - 2pi2lPl22P21lP212- 

One can verify the identity 

(12) p* ++ Det(P) = + m 2 U 13 U 23 , where 

V = P+++P222-P+++(P2++P+22+P+2+P2+2+P++2P22+) + 2p 2 ++P+2+P++2- 

Then (12) implies Det(P) > 0. 

By [5, Proposition 5.9] we can write P in terms of real vectors a;, bi 
as in (4). We obtain the identities 

U 12 = det Aidet A 2 (a 3 i + a 32 )(b 31 + b 32 ), 
U 13 = det Ai det A 3 (a 2 i + 022X621 + 622), 
U 23 = det A 2 det A 3 (a u + a 12 )(bn + 612). 

Since Ui2U\ 3 U2 3 is strictly positive, the coordinate sum of each vector 
cij, bi is nonzero. Hence our model can be equivalently parametrized by 

(13) P = sai ® a 2 <8> a 3 + tbi <S> 6 2 ® 63, 

where s,t G R and the coordinates of a iy bi sum to 1. We now show 
that (i)-(iii) ensures these parameters to be nonnegative. Note that 

U12 = det A 1 det A 2 st, 

(14) U\ 2 = det Aidet A 2 sta 31 b 3U 

■2 
12 

2 

23 

Under the specialization (13), the hyperdeterminant factors as 
Det(P) = (st det A 1 det A 2 det A 3 ) 2 . 

This gives 

Ui2Ui 3 U2 3 

St = -BeWV > °' 
and thus either s, t > or s, t < 0. By (ii), Ui 2 , U{ 21 Uf 2 have the same 
signs. Hence 031631 > and 032632 > by (14). This, together with 
the fact that \p++i] = sa 3 + tb 3 is a nonnegative vector, implies that 
a 3 , b 3 G M> if s, t > and a 3 , b 3 G 1R< if s, t < 0. The same argument 
shows that ai,bi,a 2 ,b 2 G 1R> if s,t > and ai,6i,a 2 ,6 2 G M< if 
s,t < 0. Hence, we obtain a nonnegative decomposition in (13). 

Suppose now that Ui 2 Ui 3 U 23 = 0. Without loss of generality, as- 
sume U12 = 0. Hypothesis (iii) implies U\ 2 — Uf 2 = 0. Regard 



U\ 2 = det Aidet A 2 sta 3 2b 3 2, 
and similar formulas hold for Ui 3 , U\ 3) Uf 3 and U23, U 23 , U 23 . 
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the expressions in (9) and (10) as elements in the polynomial ring 
Q[pni)Pii2, • • • 5^222]- A computation reveals the prime decomposition 

(Ui 2 ,Ul 2 ,U? 2 ) = (2x2-minorsof Flati| 23 (p)) 

fl ( 2x2-minors of Flat 2 | 13(2?) ), 

where Flatmafp) = ( 111 ^* 112 ^* 121 ^ 122 | , and similarly for Flat 2 h3(p). 

1 v ' \P211 P212 P221 V222) 1 v ' 

Hence one of these two flattenings of the tensor P G Ai e has rank 1. 

Suppose it is the first. We can find v G M> such that p^k = Vi ■ p+jk 

for every k G {1,2}. Since the 2 x 2-matrix (p+jk) can be written 

as (p+jk) = a 2 ® a 3 + ^2 ® ^3 for some a 2 , & 2 , a 3, ^3 £ ^>05 we obtain the 

desired nonnegative representation (13) by setting an = bu = V{. □ 

Theorem 1.2 tells us that the algebraic boundary of M. equals 

{Pll2P222=Pl22£>212} U {pi2l]5 222 =Pl 22 p 22 l } U {p 2 llP222=P212P221 } U 
{PlllPl22=Pll2Pl2l} U {pmP 2 12=Pll 2 £>21l} U {piHP221=Pl2lP21l}- 

Each toric cell M.^ has exactly the same algebraic boundary because 
the linear dependence constraint on double slices is void in the 2x2x2- 
case. The coordinate planes {p^k = 0} are not part of the algebraic 
boundary of M. or J\4 n . Indeed, the inverse image of {p^k = 0} under 
the parametrization lies in the boundary. But, if any of the parameters 
aij or bij is zero then the tensor P has a rank 1 slice. Hence, the set 
{pijk = 0}n.M lies in the union above. Similarly, the hyperdeterminant 
{Det(P) = 0} is not a component in the algebraic boundary of Ai. 

Example 2.3. It is instructive to look at a 3-dimensional picture of our 
7-dimensional model hA. We consider the Jukes-Cantor slice given by 



Pill 


P112 




x y 


and 


P211 


P212 




W Z 


P121 


Pl22_ 




z w 




P221 


P222_ 




y x_ 



Under this specialization, the hyperdeterminant factors as 

(15) Det(P) = (x+y+z+w)(x+y— z— w)(x— y+z— w)(x— y— z+w). 

Consider the tetrahedron {(x, y, z, w) G 1R> : x + y + z + w = 1/2} . 
Fixing the signs of the last three factors in (15) divides the tetrahedron 
into four bipyramids and four smaller tetrahedra. Inside our slice, the 
four toric cells of (6) occupy the bipyramids. Each toric cell is precisely 
the object in [8, Figure 1]. Redrawn on the right in Figure 1, its convex 
hull is the bipyramid, and it contains six of the nine edges. Any two 
of the toric cells meet in a line segment such as {x + y — z — w = 
x — y + z — w = 0, x — y — z + w > 0}. The algebraic boundary of each 
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Figure 1. Jukes-Cantor slice showing 2x2x2 tensors 
of non-negative rank < 2. Each toric cell is bounded by 
three quadrics and contains a vertex of the tetrahedron. 

toric cell consists of the same three quadrics {xy = zw}, {xz = yw} 
and {xw = yz}. Neither the three planes in (15) nor the four facet 
planes of the tetrahedron are in the algebraic boundary. □ 

3. The General Case 

Before embarking on the general proofs of Theorems 1.1 and 1.2, let 
us briefly go over an example that exhibits the general behavior. 

Example 3.1. Consider the semialgebraic set Ai of 3 x 3 x 3-tensors 
of nonnegative rank < 2. The Zariski closure Ai of Ai in IR>q 3x3 
has dimension 14 and is defined by 222 cubic equations [10, Table 3], 
namely 3 x 3-minors of the 3 x 9-matrices Flati|23(-P), Flat2|i3(-P) an d 
Flat 3 |i 3 (P). The model Ai decomposes into 108 toric cells Ai^, each 

defined in Ai by 162 quadratic binomial inequalities of the form (2). 

A quick way to generate these inequalities, for tt = (id, id, id), is to 
run the following code in the computer algebra system Macaulay2 [11]: 

R = QQ[plll ) pll2,pll3,pl21 ) pl22,pl23,pl31,pl32,pl33, 
p21 1 , p2 12 , p213 , p221 , p222 , p223 , p231 , p232 , p233 , 
p311 ,p312 ,p313 ,p321 ,p322 ,p323 ,p33l ,p332 ,p333] ; 

S = QQ[al,a2,a3,bl,b2,b3,cl,c2,c3] ; 

f=map(S,R,{al*bl*cl,al*bl*c2 J al*bl*c3,al*b2*cl,al*b2*c2,al*b2*c3, 
al*b3*cl , al*b3*c2 , al*b3*c3 , a2*bl*cl , a2*bl*c2 , a2*bl*c3 , a2*b2*cl , 
a2*b2*c2 , a2*b2*c3 , a2*b3*cl , a2*b3*c2 , a2*b3*c3 , a3*bl*cl , a3*bl*c2 , 
a3*bl*c3 , a3*b2*cl , a3*b2*c2 , a3*b2*c3 , a3*b3*cl , a3*b3*c2 , a3*b3*c3}) ; 
gens gb kernel f 
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Being 7r-supermodular means that each of the binomials in the resulting 
Grobner basis, such as p223*p312-p212*p323, must be non-positive. 

The algebraic boundary of A4 has nine irreducible components, cor- 
responding to the nine slices of P. It is instructive to see how our 162 
hypersurfaces, like {^223^312 = P212P323} H M., break into these com- 
ponents. Each individual toric cell A4 n has 18 irreducible components 
in its algebraic boundary: now also the 9 double-slices kick in. The 
intersection of all 108 toric cells is the Segre variety of rank 1 tensors, 
whose reverse lexicographic Grobner basis we identified with (2). □ 

A marginalization of P is any tensor obtained from P by summing 
all slices for some fixed indices. For instance, the 2x2-matrix (pij+) 
is a marginalization of the 2x2x2-tensor P = (pijk)- The following 
lemma, whose proof is delayed, will be useful in proving Theorem 1.1. 

Lemma 3.2. All marginalizations of a supermodular tensor are su- 
permodular, and ditto for e- supermodular with e = (id, id, . . . , id). In 
addition, all flattenings of a supermodular tensor are supermodular. 

In this lemma, and in the remainder of the paper, we use the term 
flattening to include all tensor flattenings, not just the matrix flatten- 
ings described in the introduction. We now prove our first main result. 

Proof of Theorem 1.1. Suppose first that P has nonnegative rank < 2. 
Then P has the form (1) with aj,6j G R> . As tensor rank cannot 
increase under flattening, we conclude that P has flattening rank < 2. 

Consider the di x 2-matrix with columns eij,&j. By swapping rows 
we can make all 2 x 2-subdeterminants of these n matrices (a*, bi) non- 
negative. But swapping rows in these matrices corresponds to acting 
on P by n, where tcP : = [p^-im] for i = (ix,...,i n ) and 7r _1 (i) = 
(tti (ii), ■ ■ ■ ,7r~ 1 (i„)). Since P G Ai e if and only if ttP G Ai^, it suf- 
fices to prove the following: if P has the form (1) with a^bn > aubik 
for every % and all k < I then P G M. e - 

To prove this we define an auxiliary 2 x d\ x • • • x d n tensor P by 

Plil—i„ = a \i\ a 2i2 ' ' ' a ni n &nd P2i 1 --i n = ^lii^2i 2 ' ' ' ^ni n ■ 

We claim that P is e-supermodular. For this, we need to check that 

(16) Pioh—inPjoji—jn — Pkoki—k n Ploh—l n i 

for all i r ,j r such that k r = min{i r , j r }, l r = max{i r , j r }. This holds 
with equality if io = jo- If *o 7^ jo we have two cases to consider, and 
our claim (16) is equivalent to the inequality 

max{aij 1 bu 1 ■ ■ ■ a n j n b n i n , au 1 bij 1 ■ ■ ■ a nin b n j n } < ai^bi^ ■ ■ ■ a n kn^ni n - 
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Our assumption on the 2 x 2-subdeterminants of (a r , b r ) ensures 
m&x{a rjr b rir , a rir b rjr } < a rk b rlr for every r G [n\. 

This gives the desired inequality, and therefore P is e-supermodular. 
But P is a marginalization of P because v%x-i n = Pih—in +P2ii-i„, so 
Lemma 3.2 then implies that P is e-supermodular. 

For the converse, consider any supermodular d\ x • • • x d n tensor 
P of flattening rank < 2. Let Fi be the flattening of P given by the 
partition {i}, [n]\{i}. Suppose rank(Fj) < 2 for some i, say i — 1. 
Then P = v £g> P' for some v G 1R>q and P' = \p+i 2 -i n ]- By Lemma 
3.2, the marginalization P' is supermodular with flattening rank < 2. 
By repeated application of this argument, we may reduce to tensors P 
whose di x (d\ ■ ■ ■ di-\di + \ ■ ■ ■ d n )-flattenings Fi all have rank exactly 2. 

We next reduce to tensors of format 2 x ■ ■ ■ x 2. Let P be a su- 
permodular d\ x • • • x d n tensor all of whose flattenings are of rank 2, 
and U C R di the span of the columns of a flattening Fj. Two suitable 
columns of Fi give a nonnegative basis {a, b} of Lj. We modify this 
basis to {a', 6'} so that, after permuting entries, it is nonnegative and 

a = (1,0,*,...,*), 
6' = (0,1,*,. ..,*). 

To obtain this nonnegative basis first set a" = a — tb, using the maximal 
t for which a" is nonnegative. Then set b" = b — sa" with the maximal 
s for which b" is nonnegative. The vectors a", b" each have an entry 
of in a position where the other does not. Rescaling so the non-zero 
entries in these positions become 1, and permuting entries to bring 
these positions to the first two, we obtain the desired a', V . 

Now every column of Fi is in the nonnegative span of a' , b' . More 
concretely, we have Fi = Cf ■ F/, where Ci has rows a',b', and F[ is 
the first two rows of F{. On tensors, this is expressed by 

P = P'*i 

where P' is the double slice of P with zth index in {1, 2} and P' *j Cj 
denotes the linear action of Ci on the zth index of P'. Applying this 
construction in each index we find (after suitable relabelings) that 

(17) P = P *(Ci,...,C n ), 

where Pq is the 2x • • • x2 subtensor of P obtained by restricting all 
indices to {1, 2}, and the 2 x ^-matrices Ci are real and nonnegative. 

Our hypotheses ensure that P is supermodular with all flattening 
ranks 2. Moreover, if P Q has nonnegative rank 2, then it follows from 
equation (17) that P also has nonnegative rank 2. Explicitly, if Pq = 
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a± <g> • • • <g) a n + h\ ® • • • <g> b n is a nonnegative decomposition, then 
P = ai <8> • • • <8> a n + b\ ® ■ ■ ■ ® b n with 5j = ctjCj, 6« = 6jCj nonnegative. 

It remains to show the result for 2 x • • • x 2 tensors. Let P' denote 
the 2x2 x2"" 2 flattening of P from the tripartition {1}, {2}, [n]\{l, 2}. 
By Lemma 3.2, P' is supermodular. By Proposition 2.2, each 2x2x2 
subtensor of P' has nonnegative rank < 2. The argument of the last 
three paragraphs implies that P' itself has nonnegative rank < 2, so 

P' = oi <g> a 2 <S> a 3 + bi <8> b 2 <8> 63, 

with ai, 02,61,62 G 03,63 G JR>" 2 . The matrices A = (ai,&i) T 

and -B = (a 2 , b 2 ) T are invertible, by our assumptions on the 2 x 2™ _1 
flattening ranks of P. Acting on the tensor P by their inverses, we get 

P = ei ® ei ® iVi + e 2 ® e 2 ® A^ 2 , 

where Ni,N 2 are nonnegative tensors whose vector flattenings are 03, 63. 

Consider any bipartition A, B of {3, ... , n}. The 2l A l +1 x 2l B l +1 flat- 
tening of P using the bipartition {1}Uv4, {2}UB is block-diagonal, with 
blocks given by A\B flattenings of N U N 2 . This 2 |A|+1 x 2 |B|+1 matrix 
has rank < 2, so either both flattenings of iVj have rank < 1, or one iV, 
is zero. But iVj = is impossible since that would mean some 2 x 2" _1 
flattening of P has rank 1. Hence the A\B flattenings of Ni,N 2 have 
rank 1. Since A, B were arbitrary, both iV, have (nonnegative) rank 1. 
Consequently, P has nonnegative rank 2, and so does P. □ 

It remains to prove Lemma 3.2. We shall use the Four Function 
Theorem of Ahlswede and Daykin [1], here presented in a special case: 

Proposition 3.3. [Ahlswede-Daykin] Fix n > 2 and a nonnegative 
diX ■ ■ ■ xd n -tensor P = [pi 1 ...i n \- For any collection C of indices i = 
(ii, . . . , i n ) in [di] x • • • x [d n ] define pc = J2iecP l - Suppose that 

(18) Pi ■ Pj < Pivj " PiAj for any two indices i, j, 

where V, A are join and meet operations that gives [di] x ■ ■ ■ x [d n ] a 
lattice structure. Then for any two collections C, C , we have 

Pc -Pc> < Pcvc -Pcac, 
where CV? = {iVj : i G C, j G C'} and C AC = {i Aj : i G C, j G C'}. 

Proof of Lemma 3.2. Let P be a supermodular d±X ■ ■ ■ x<i n -tensor. For 
the first assertion, it suffices to show that P' = [p+i 2 -i„] is supermod- 
ular. The general statement for marginal tensors follows by induction. 

If P is 7r-supermodular, define the lattice structure on [di] x • • • x [d n ] 
by taking k = i Aj if and only if 7r(k) is the coordinatewise minimum of 
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7r(i) and 7r(j). Similarly, 1 = iVj if and only if 7r(l) is the coordinatewise 
maximum of 7r(i) and vr(j). Fix i', j' G [0I2] x • • • x [d n ] and set 

C = {(«!,!'): he[di]}, C = : ZiG[d x ]}. 

We have pc = J2iecP l = P+ 1 ' anc ^ Pc = P+i'- The tensor nP = (p^-im) 
is e-supermodular. Proposition 3.3 now gives 

P+i'- P+i' < P+(i'A)') " P+(i'Vj')- 

This means that P' is 7r'-supermodular, where it' = (112, . . . , ir n ). 

We now prove that every flattening of P is supermodular. Let Q = 
[Qai-a r ] be a flattening of P corresponding to the partition A±, . . . , A r of 
{1, . . . , n}. Let hi = Ylj^Ai then ck = (ai, . . . , a r ) G [hi] x • • • x [h r ]. 
Without loss of generality we can assume that a« indexes elements of 
rijeAj^i] ordered lexicographically. Every q a is equal to pi for some 
i, so that each a. corresponds to a unique i. Since P is supermodular, 
there exists n = (tti, . . . , 7r„) such that for every i,j we have piPj < 
PiAjPivj, where i A j and i V j is as defined in the previous paragraph. 

Define now a A (3 and a V (3 to be the r-tuples corresponding to 
i Aj and iVj. The permutation it induces the corresponding r-tuple of 
permutations n = . . . , n r ) such that 7r(i) = n(oc). By construction, 
we have p^pp < PaApPavp, where it (a. V /3) < n(a V /3). This implies 
that Q is 7r-supermodular. □ 

We now prove the second theorem stated in the Introduction. 

Proof of Theorem 1.2. The formula (1) defines a polynomial map 

. jj^2(di+d2H \-dn) y jj^di x<fe x •■■ xd„ 

such that Ai = 0(1R>q 1+ dn ' ) ) is the set of tensors of nonnegative rank 
< 2. We modify the domain by assuming the coordinate sums of all a, 
and hi are 1, while adding two mixture parameters s, t as in (13). This 
does not change the image, but makes the map generically 2-to-l. More 
specifically, <fi is 2-to-l on the open set where st 7^ and each pair aj, 6« 
is linearly independent. Since this open set intersects the coordinate 
hyperplane {a^ = 0} (or {bij = 0}), the map is generically finite 
on that hyperplane. Hence the closure of the image 4>({aij = 0}) is 
an irreducible subvariety of codimension 1 in Ai. Moreover, in any 
neighborhood of a point on {a^- = 0} there are points with a^- < 
that are not mapped into the interior of Ai. Indeed, generically 
the fiber containing such a point only contains its image under label 
swapping, and thus all points in the fiber have a negative coordinate. 
Thus 4>({ciij = 0}) is a component of the algebraic boundary of Ai. 
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By restricting to open subsets where the signs of all 2 x 2-minors of 
the matrices (a*, bj) are fixed, we see that 4>({a,ij = 0}) is also a compo- 
nent in the algebraic boundary of Ai n . Additional pieces of the bound- 
ary of U n are the quadrics {aijbik = dikhj}, on whose general points 
the map is also 2-to-l. Therefore the varieties <p{{dijbik = cukbij}) 
are irreducible of codimension 1 in Ai^, and, by the same argument as 
above, they are also components of the algebraic boundary of M.^. 

We next argue that there are no components in the algebraic bound- 
ary of Ai or A^tt other than the two types we just identified. This 
follows from Theorem 1.1. Let P G dAi-n- Consider the binomials 
Pi^h-iJPhh-in ~ Pkxki-kJPhh -in tnat correspond to facets of the poly- 
hedral cone of supermodular functions. For such a facet binomial, the 
indices in the four appearing unknowns p, agree in all but two of the 
positions. All other binomials (2) admit representations such as (8). 
The expansion of a facet binomial into parameters , by factors into 
coordinates and 2 x 2-determinants as in (7). Hence, at the two points 
in (p~ 1 (P), one of these factors must vanish, and this implies that P 
lies on one of the hypersurfaces we already identified above. 

We finally identify 4>{{a,ij = 0}) and 4>{{a,ijbik = aikhj}) with the 
rank loci described in the statement of Theorem 1.2. If the coordi- 
nate ciij vanishes then the j-th slice of P in the i-th dimension drops 
its rank from < 2 to < 1. Likewise, if a^b^ = Q-ikbij, then the jth and 
kth slices of P in dimension i becomes linearly dependent. Hence the 
irreducible components of the algebraic boundaries of Ai and Ai-* are 
uniquely characterized by lying in the following two types of rank loci: 

(a) the variety of tensors P of border rank < 2 such that a partic- 
ular slice has border rank < 1; 

(b) the variety of tensors P of border rank < 2 such that a partic- 
ular double slice is linearly dependent. 

This completes the proof of Theorem 1.2. □ 

We believe that the rank loci in (a) and (b) are irreducible varieties, 
and that their prime ideals are generated by the relevant sub determi- 
nants of format 2x2 and 3x3. At present we do not know how to 
prove this. A similar issue for tree models appears in Conjecture 4.2. 
For the case of Example 3.1, we proved irreducibility by computation: 

Example 3.4. The variety Ai of 3 x 3 x 3 tensors of border rank < 2 has 
dimension 14 and degree 783. Using Macaulay2 [11], we verified that 
both (a) and (b) define irreducible subvarieties of dimension 13. The 
variety (a) has degree 882, and its prime ideal is minimally generated 
by 9 quadrics and 187 cubics. The variety (b) has degree 342, and its 
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prime ideal is minimally generated by 36 quadrics and 90 cubics. All 
ideal generators can be chosen from the relevant subdeterminants. □ 

One may ask how efficiently the model membership can be tested. 
The number of facets of the submodular cone is a polynomial in the size 
of the tensor, and each facet inequality involves precisely four of the 
unknowns. Hence supermodularity for positive tensors can be tested in 
polynomial time. For instance, a 2 x 2 x • ■ ■ x 2-tensor has N = 2 n cell 
entries, and the facets correspond to the 2-faces of the n-cube (see the 
proof of Theorem 1.2), of which there are only n(n — l)2 n_3 = 0(N 1+e ). 

4. Binary Tree Models 

In this section we study the extension of our results to the general 
Markov model M.t on a phylogenetic tree T with binary states [2, 3, 6, 
12, 21]. The special case when T is a star tree, with only one internal 
node, corresponds to 2 x 2 x • • • x 2-tensors of nonnegative rank < 2. For 
arbitrary trees T, Steel and Faller [20] showed that distributions in M. T 
are supermodular, by a marginalization argument as in Lemma 3.2. 

We assume that T has n > 3 leaves, E is the set of edges of T, 
and one of the \E\ — n + 1 internal nodes is the root of T. We specify 
each probability distribution P in the model M. T by a nonnegative 
root distribution tt G R> , together with a 2 x 2 Markov matrix M e 
for each edge e, directed away from the root. The entries of tt and 
of each row of each M e sum to 1. These choices determine a point 
9 = (vr, (M e ) ee£ ) in the cube 6 = [0, l] 2|s|+1 . That cube serves as 
the domain for the model parametrization <p : O -» M. T c IR>q 2x "' x2 . 
This can be found in explicit form in [2, Equation (1)]. The map (f) is 
locally identifiable. To be precise, each general fiber consists of 2l- B l _n+1 
points, corresponding to label swapping on the internal nodes. Hence 
our binary tree model Air = <K©) is a compact semialgebraic set of 
dimension 2\E\ + 1 inside the probability simplex of dimension 2 n — 1. 
It is known that Air is independent of the choice of the root node. 

The prime ideal that defines the Zariski closure M. T is known. It is 
generated by the 3 x 3-minors of all flattenings of P that are compatible 
with T. Here, a split (A, A c ) of [n] is compatible with T if the intersec- 
tion of any path between two leaves in A with any path between two 
leaves in A c is either empty or just one internal node. This was first 
shown set-theoretically for trivalent trees by Allman- Rhodes [2]. The 
ideal-theoretic statement for arbitrary trees T is seen by combining the 
result of Draisma-Kuttler in [6] with the result of Raicu in [18]. 

Our main result of this section concerns the algebraic boundary of 
the general Markov model M.t inside the phylogenetic variety M.t- 
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Theorem 4.1. The algebraic boundary of the binary tree model A4t 
has n+\E\ irreducible components, two for each of the n pendant edges, 
and one for each of the \E\ — n internal edges. The components are 
closures of images of facets of the cube G, as described below. 

The components of the algebraic boundary of Air are as follows: 

(1) For each pendant edge e with leaf £, fix one row of the 2x2- 
matrix M e . (The other row gives the same two components.) 
Nonnegativity of either entry determines a facet F of the cube 
0. Then 4>(F) is a component. It has the following equational 
description inside Air- If the internal node on e is r-valent, 
it gives a partition [L\ = {£}, L 2 , . . . , L r ) of [n]. Flatten P 
accordingly to a 2 x 2^ x • • • x 2l L ' l tensor. The rank of the 
1 x 2 |L21 x ■ ■ ■ x 2 |Lr| slice selected by F drops to < 1 on (p(F). 

(2) For each internal edge e, fix any one entry of the 2 x 2-matrix 
M e . (The other three entries give the same component.) Non- 
negativity of that entry determines a facet F of the cube O. 
Then <f>(F) is a component. It has the following equational de- 
scription inside AAt- Let T[e] be the tree obtained from T by 
contracting e. For either matrix flattening of P that is compat- 
ible with T[e] but not with T, the rank drops to < 3 on 4>{F). 

At present we do not know whether the equational descriptions above 
(in terms of tensor rank) are enough to cut out the codimension 1 sub- 
varieties <f)(F) of Air- For this, it would suffice to prove the following: 

Conjecture 4.2. The rank varieties in (1) and (2) are irreducible. 

We have a computational proof of Conjecture 4.2 in the smallest 
non-trivial case, the trivalent tree on 4 taxa, which we discuss next. 

Example 4.3. Let n = 4 and T the trivalent tree with split 12 134. 
The phylogenetic variety lives in P 15 and it has dimension 11: 



The model Air is composed of eight 11-dimensional cells At^, where 
7r = (7ri,7r2,7r 3 ,7r 4 ). As before, Xid,id,id,id = A4(i 2 ),(i2),(i2),(i2)- These 
cells are glued together along lower- dimensional models corresponding 
to forests obtained by deleting edges of the tree. For instance -Mid,id,id,id 
is glued to -Mid,id,(i2),(i2) along the model of two independent 2-leaf 
trees. It is also glued to •Mid,id,id,(i2) along a model of a 3-leaf tree and 
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an independent leaf, and similarly to 3 other cells. Finally, it is glued 
to the remaining cells -M(i2),id,(i2),id and A4(i2),id,id,(i2) along even more 
degenerate models, of a forest with one 2-leaf tree and two singleton 
leaves. All eight cells intersect in the model of four independent leaves. 
The various strata correspond to P 3 xP 3 , P 7 xP 1 , P 3 x(P x ) 2 and (P 1 ) 4 . 

The algebraic boundary of M. T has eight irreducible components of 
type (1), such as 



The 2x2-minors of (20) and 3x3-minors of (19) generate a prime ideal. 

The ninth component of dAir comes from the internal edge and is 
of type (2). It is defined by the 4 x 4-determinant of either of the 
two flattenings other than (19). These two determinants are equal and 
irreducible on A4t, so they give the prime ideal of that component. □ 

To prove Theorem 4.1, we consider the singular locus S i ng of the 
parametrization 0. By definition, 6 sing is the closed subset of the cube 
G where the rank of the Jacobian matrix of </> drops to 2\E\ or below. 

Lemma 4.4. S i ng is the subset of points in where either the root 
distribution tt has a zero entry, or some Markov matrix M e is singular. 

Proof. A tree T n with n leaves is obtained by attaching a cherry to a 
leaf i of an in — l)-leaf tree T n _i. Assuming the matrices M e on T„_i 
are non-singular and n has non-zero entries, then the distribution for 
T n _i flattens on the edge incident to i to a 2 x 2 n_2 matrix A of rank 2. 
Let ax, hi be the rows of A, and a 2 , 62 and (X3, 63 the rows of the matrix 
parameters on the edges of the cherry. Then the distribution for T n , 
appropriately flattened, is a\ ® a 2 ® 03 + b\ <g> b 2 ® b^. 

We next show the map 

if; : (ax, 61, a 2) b 2) a 3 , 63) h-> ai <g> a 2 <g> 03 + h <g) b 2 <8> h 

where the entries of a 2 ,b 2 ,a3,bs sum to 1, is non-singular precisely at 
points where all pairs ai, hi are linearly independent. That if) is singular 
at points where some pair ai, bi is dependent is straightforward. To 
show the rest of this claim, we allow arbitrary real entries in the vectors, 
to take advantage of a group action. 

Let G be the subgroup of GL(2 n ~ 2 ) x GL(2) x GL{2) consisting of 
matrix triples (gi, g 2 , g%) where the rows of g 2 and ^3 sum to 1. The 
group G acts on both the domain and range of if>, and intertwines as 
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Hence the Jacobian matrix of ?/> has constant rank on each orbit. But 
the orbit of any point with all pairs a%, hi linearly independent is dense 
in the domain. Thus if ip were singular at such a point, it would be 
singular everywhere. Since ip is generically 2-to-l, that is impossible. 

Note that the statement of the lemma for the 3-leaf tree follows from 
the previous paragraphs. Building the tree T n inductively from T n _\ 
writes the Jacobian of ^ as a product of block matrices of smaller 
Jacobians. From this we see that S i ng consists of points where either 
tt has a zero entry, or some M e is singular. □ 

Lemma 4.5. If 9 G © S ing ; then the fiber of 9 intersects the boundary 
of Q, i.e. there exists 9' G <90 with <fi{9') = (f)(9). Moreover, 9' can be 
found in a facet of where some entry of a Markov matrix is zero. 

Proof. For a 3-leaf tree, rooted at the internal node, consider the pa- 
rameters 9 = (tt, Mi, M 2 , M 3 ) G ©sing- If = 0, then we may replace 
row i of any or all Mj with (1,0) to obtain 9'. Otherwise, suppose 
M3 is singular yet there are no zeros in the parameters. Define 9' by 
tt' = vrMi, M[ = the identity matrix, M' 2 = diag(7r')" 1 M 1 T diag(vr)M 2 , 
and M3 = M3. One checks that <p{9') = <p(9), and M[ has a zero entry. 

The result is derived inductively for larger trees, by viewing them as 
built from 3-leaf trees by attaching cherries. □ 

Proof of Theorem 4-1- Points in Int(0)\O sing must map to points in 
the relative interior of the model M.t- Thus the boundary of M.t is a 
subset of 0(90) U 0(© S i ng ). By Lemma 4.5, this is contained in 0(90). 

To see that each of the components listed is a boundary component, 
we must show they have codimension 1 in the model, and a Zariski 
dense subset of points in them are limits of points outside the model. 
Since the complement of S i ng intersects these facets of O in non-empty 
open sets, the codimension is as needed. Since all elements of a fiber of 
the parameterization which contains non-singular points are related 
by label swapping, even when the map is extended outside O, one sees 
that non-singular points outside O cannot be mapped into the model, 
yet they are mapped arbitrarily close to the claimed component. 

We have discussed all but two of the 4\E\ + 2 facets of O. The 
remaining two facets, where an entry of the root distribution tt is 0, 
contain only elements of S i ng , by Lemma 4.4. By Lemma 4.5, they lie 
in fibers with points where some entry of a Markov matrix is zero. Thus 
they are mapped into a component of the boundary already identified. 

It remains to be shown that the equational descriptions given in (1) 
and (2) are valid on the respective components of the boundary of M.t- 
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For a pendant edge e as in (1), we can assume that the root of the 
tree is located at the non-leaf end of e. The sets Lj span subtrees that 
intersect only at the root, and for each j there is a 2 x 2' L ^' matrix 
Aj, dependent only on the Markov matrices on edges of the subtrees, 
which expresses the joint probabilities of states at the leaves in Lj, 
conditioned on the root. In particular, A\ = M e . Denoting the rows of 
Aj by aj, bj, the r-dimensional flattening of our distribution is 

7Tx • ai ® a 2 <S> • • • <8> a r + tt 2 • &i <8> b 2 <8> • • • <8> b r . 

If an = (or bu = 0) then the ith slice in the first index has rank < 1. 

For an internal edge e as in (2), assume the root of the tree is located 
at one end, and the Markov matrix on the edge is M e , with rows a e ,b e . 
Let Li, L 2 and L 3 , L 4 be the leaves of the subtrees attached to the 
respective ends of e. Then the 2' LlUi3 ' x 2' L2Ui4 ' matrix flattening 
incompatible with T can be expressed as 

(21) A T diag(7r 1 a e , n 2 b e )B, 

where A,B are 4 x 2l LlUL3 l and 4 x 2l L2UL4 l matrices, respectively The 
entries of A depend on the parameters on the subtrees on L\ and L3, 
while those of B depend on the parameters on the subtrees on L 2 and 
L4. Thus if M e has a zero entry, then the 4 x 4-matrix diag(7r 1 a e , 7i 2 b e ) 
is singular, and hence the flattening (21) has rank at most 3. □ 

Several recent works found semialgebraic descriptions of the 2-state 
general Markov model on trees that is considered here. In [21] a differ- 
ent coordinate system is used, but [3] follows the same framework as 
this paper. Although some of the inequalities given in [3] hint at the 
form of the algebraic boundary determined in Theorem 4.1, those in- 
equalities are considerably more complicated than our description here. 
While the inequalities provide tests for model membership, the relative 
simplicity of the algebraic boundary is expected to be advantageous for 
other purposes, such as understanding the geometry of log-likelihood 
functions over Air, and studying the limit behavior of iterative meth- 
ods for parameter estimation such as Expectation Maximization (EM). 

5. Towards Higher Rank 

There are formidable obstacles to extending our results to tensors of 
rank r > 2. First of all, we do not know how to generalize the super- 
modular constraints. Second, we run into problems of non-identifiability, 
even in the case of matrices (n = 2). Recall also (e.g. from [7, Example 
4.1.2]) that a nonnegative matrix of rank 3 need not have nonnegative 
rank 3. The topological analysis given by Mond et al. [15] illustrates 
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well the difficulties involved in obtaining a characterization of the semi- 
algebraic set of d\ x d 2 matrices of nonnegative rank < 3. 

On the other hand, for tensors of dimension n > 3, rank decomposi- 
tions are often identifiable when r is small relative to di, d 2 , ■ ■ ■ , d n . In 
such situations, when the model is identifiable, one might hope for re- 
sults similar to Theorems 1.2 and 4.1. However, a third obstacle arises: 
in order to characterize algebraic boundaries, one needs a version of 
Lemma 4.4 for the singular locus S i ng of the model parameterization (f). 

In what follows, we illustrate these issues for two rank 3 examples. 

Example 5.1. Consider the set Ai of 3 x 3 x 2 tensors of nonnegative 
rank < 3. This is a smallest format for which rank 3 decompositions 
are generically unique, up to label swapping. Normalizing the tensor 
entries to sum to 1, we obtain Ai as the image of the map 

<p : -)■ A 17 , (7r; Oi, a 2 , a 3 ; h,b 2 , b 3 ; c 1 , c 2 , c 3 ) !->■ 

n i ■ ai ® a 2 <8> a 3 + tv 2 ■ h ® b 2 <g> 63 + ^3 • c i ® c 2 <E> c 3 , 

where 7r £ M> , a^, 6j, q G 1R> for i = 1,2, and 03, 63, C3 G 1R> all have 
coordinate sum 1. The domain is the polytope = A 2 x(A 2 x A 2 x Ax) 3 . 
The facets of are given by parameters being 0. This map is generi- 
cally 6-to-l, so 0(0) = Ai is full-dimensional in the simplex A17. The 
Zariski closure Ai is the entire projective space P 17 of 3x3x2 tensors. 
The algebraic boundary dAi has eight irreducible components: 

(a) Two components <p{{a 3k = 0}) = 4>({b 3k = 0}) = <p{{c 3k = 0}), 
for k = 1,2, given by the 3x3-slice P k = \p** k ] having rank < 2. 

(b) Three components given, for i = 1,2,3, by the 3 x 3-matrix 
-Pi ' (-P2) -1 having an eigenvector with zero i-ih coordinate. 

(c) Three components given, for j = 1, 2, 3, by the 3 x 3-matrix 
Pi ■ (P 2 )~ T having an eigenvector with zero j-th coordinate. 

The two components (a) are the cubic hypersurfaces given by the de- 
terminants of Pi and P 2 . The six components (b) and (c) are hyper- 
surfaces of degree 6. For instance, the polynomial K that defines the 
(b) component (p({a 13 = 0}) = 4>({bi 3 = 0}) = 0({ci 3 = 0}) equals 

K = PlllP212P32lP332 - 2pillP212P32lP322P33lP332 + PlllP2\2P 322 P 33 l 

-PlllP222P31lP32lP33 2 + PlllP222P311i?322P331?>332 ~ PlllP222P312P322j>33i 
+P111P222P312P321P331P332 + P111P232P311P321P322P332 ~ P\\2P2\\P 32 \P 332 
-PlllP232P31lP322^331 + P111P232P312P321P322P331 ~ PlllP232P312P 32 iP332 
+2P112P211P321P322P331P332 - Pll2P21lp| 22 p| 31 + Pll2 M 31lP32lP§ 32 
-P112P221P311P322P331P332 ~ Pll2P22lP312P32lP33lP332 + Pll2P22lP312P322P| 3 i 

-P112P231P311P321P322P332 + Pll2y>23lP31lP 322 P331 + Pll2P23lP312p| 2 iP332 
-pil2P23iy312P32lP322P331 ~ Pl2lP212P31lP32lp| 32 + P121P212P311P322P331P332 
+P121P212P312P321P331P332 - P121P212P312P322P331 + Pl2lP222P 3 l\P 332 
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-2pi2lP222P31lP312P33lP332 + Pl2lP222Pll 2 Phl ~ PY2\P2Z2p\\\PZ22PZZ2 
+Pl2lP232?>31lP>312£ , 32l£ , 332 + Pl2lP232P31lP312i?322P331 ~ Pl2lP232P|i 2 P32lP331 
+P122P211P311P321P332 - P122P211P311P322P331P332 - P122P211P312P321P331P332 

+P122P211P312P322P| 3 1 ~ WmRBIpIi tf&a + 2pi22P22lP31lP312P33lP332 

-P122P221P312P331 + Pl22P23lP311^322P332 ~ Pl22?>23lP31lP312P32lP332 

-Pl22P23lP31iy312P322P331 + Pl22P23lP3i 2 P32lP331 + P131P212P311P321P322P332 

-Pl3lP212P31lPl 2 2P331 ~ Pl3lP212P312P| 2 lP332 + Pl3lP212P312P32lJ>322j>331 
2 

— Pl3lP222P3nP322P332 + P131P222P311P312P321P332 + P131P222P311P312P322P331 
-P131P222P|1 2 P321P331 + Pl3lP232p|nP| 2 2 ~ 2pi 3 ip 2 32P31lP312P32lP322 
+P131P232P112P321 ~ Pl32P21l£>31lP32lP322f>332 + Pl32P21lP31lp! 2 2P331 
+Pl32P21lP312P32lP332 - P132P211P312P321P322P331 + Pl32P22lp|nP322P332 
-P132P221P311P312P321P332 ~ Pl32P22lP31lP312f>322P331 + Pl32P22lP3i 2 P32lP331 
-Pl32P23lPlllPi 2 2 + 2pi32P23lP31lP312P32lP322 ~ Pl32P23lPll 2 phl • 

This polynomial was found using the reduced Kalman matrix in [19, 
equation (1.5)]. Under the parametrization, this expression factors as 

K = k\k\k\ dl3&13Cl3(a 3 l&32 - a32&3l) (a 3 lC 3 2 - a 3 2C3i)(&3iC 3 2 - &32C31) 

x det[ai, b\, ci]det[a 2 , b 2 , c 2 ] 2 



To prove that there is nothing else in dAi, we proceed as in Lemma 
4.5. We examine the Jacobian of 0, which has rank 17 at generic points 
of 6. Using symbolic computation, we find that its singular locus S mg, 
where the rank drops, decomposes into three types of components: 

(1) points with ir having a zero entry, at which the rank of the 
Jacobian is generically 12, 

(2) points with two of 03, 63, C3 equal, at which the rank of the 
Jacobian is generically 15, 

(3) points with a\,bi,c\ (or with a 2 ,b 2 ,c 2 ) linearly dependent, at 
which the rank of the Jacobian is generically 14. 

We now show that every point of S i ng lies in a fiber of (f> that intersects 
the boundary of the polytope O. For singular points of type (1), if say 
7i"i = 0, one may replace an tjj with any other vector to obtain another 
point in the fiber, so this is clear. For type (2), if say 63 = C3, then 

vr 2 • bi ® b 2 <S> h + ^3 • c\ ® c 2 <S> c 3 = A <g> b 3 

for a 3 x 3 matrix A of nonnegative rank 2. Since one can find a 
nonnegative rank 2 decomposition of A with zeros in some vector entry, 
we can construct the desired boundary point in the fiber. 

For type (3) singular points we argue as follows. Suppose P is the 
image of parameters where a 2 ,b 2 , c 2 are dependent. Let d be a nonzero, 
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nonnegative vector in the span of a i, b\ and consider the line of tensors 

B(t) = P - tt 3 • (d + td) ® c 2 <g> c 3 

= ai ® a 2 <g) a 3 + n 2 ■ b\ ® b 2 <E> o 3 - tn 3 ■ d ® c 2 ® c 3 . 

The sets {ai,&i,d}, {a 2 ,5 2 ,c 2 } and {03,63,03} are dependent, so the 
three matrix flattenings of B(t) have rank < 2. Thus for all t, the 
tensor B(t) has border rank < 2. Also, P(0) has nonnegative rank 2. 

Since P(t) fails to have nonnegative rank 2 for t ^> 0, there exists 
to > such that B(to) lies on the boundary of the tensors of non- 
negative rank 2. By Theorem 1.2, B(to) has a nonnegative rank 2 
decomposition with a zero coordinate in some parameter vector. Since 

P = B(t ) + tt 3 (ci + t d) <g) c 2 <g> c 3 , 

the tensor P has a nonnegative rank 3 decomposition with a zero pa- 
rameter. Hence P lies in one of the eight varieties seen in (a),(b),(c). 

We note that a distribution P with invertible slices Pi lies in Ai if and 
only if all eigenvalues and eigenvectors of the 3 x 3- matrices Pi • (P 2 ) -1 
and P[ ■ {P 2 )- T are nonnegative. Here, one should be able to pass to 
the closure and infer a nice semialgebraic description of Ai. □ 

Example 5.2. Let Ai be the set of 2x2x2x2 tensors of nonnegative 
rank < 3. As in the previous example, we normalize tensors to have 
entries summing to one. This model is not identifiable: the generic fiber 
of its stochastic parametrization is a curve. Facets of the parameter 
polytope 6 = A 2 x (Ai) 12 are mapped into subsets of the model Ai 
that are Zariski dense in Ai. We note that the 13-dimensional variety 
Ai is a complete intersection of degree 16 = 4 • 4 in P 15 . It is defined 
by the determinants of any two of the three 4 x 4-flattenings of P. 

Components of the algebraic boundary dAi might now be obtained 
from codimension 2 faces of the polytope G. For instance, write 

P = 7Tiai <g) a 2 g) a 3 ® a 4 + 7r 2 &i <g> b 2 <g) 63 <g> 64 + 7r 3 Ci <g) c 2 <8> c 3 ® c 4 , 

and consider the face {an = 6 22 = 0} of 6. Then 0({an = b 22 = 0}) is 
suspected to be a component in dAi. This variety has dimension 12 and 
degree 56 in P 15 . It is defined, as a subscheme of Ai, by 55 polynomials 
of degree 8 in the 16 unknowns Pijki- The smallest of these degree 8 
polynomials has 96 terms, and we shall resist the temptation to list 
them. The semialgebraic geometry of Ai deserves further analysis. □ 
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